1/1 


RD-Aise  871  fl  MODEL  FOR  GENERRLI2RTI0N  RND  SPECIFICRTION  BV  SINGLE 
NEURONSCU)  BROUN  UNIV  PROVIDENCE  RI  CENTER  FOR  NEURRL 
SCIENCE  P  MUNRO  @8  FEB  85  TR-2e  Ne0dl4>81-K-013G 

F/G  S/16 


UNCLRSSIFIED 


NL 


AD-A150  871 


I  nepomt  NUMScn 


KbKOKI  DOCUMfcNTAllON  HAUt  _ 


1.  GOVT  ACCESSION  NO.  )  RECINICnT'S  CATALOG  NUMGCN 


BEFORE  COMPLETING  FORM 


4.  title  fAnd  SubtIN*) 


A  Model  for  Generalization  and  Specification 
by  Single  Neurons 


S.  TYPE  OF  REPORT  A  PERIOO  COVEREO 

Technical  Report 


S.  PERFORMING  ORG.  REPORT  NUMRER 


T.  AuTMORfaJ 


Paul  Munro 


S.  CONTRACT  OR  GRANT  NUMRERfbJ 

N00014-81-K-0136 


t.  PERFORMING  organization  NAME  AMO  AOORESS 

Center  for  Neural  Science 
Brown  University 

Providence,  Rhode  Island  02912 


II.  CONTROLLING  OFFICE  NAME  AND  AOORESS 

Personnel  and  Training  Research  Program 
Office  of  Naval  Research,  Code  442PT 
Arlington,  Virginia  22217 


12.  REPORT  DATE 

February  8,  1985 


U.  NUMBER  OF  pages 

35 


Ia:  monitoring  agency  name  a  AOORESS^I/  Inm  Cantntunt  Otllc»)  I  IS.  SECURITY  CLASS,  (at  Ihia  rmpatt) 


UNCLASSIFIED 


isa  declassification/ downgrading 
SCNEOULE 


16.  distribution  STATEMENT  (at  Ihia  Rapatl) 


Approved  for  public  release:  distribution  unlimited.  Publication  in 
part  or  in  whole  is  oermitted  for  any  purpose  of  the  United  States 
Government . 


17.  distribution  statement  (at  IHa  ahaltmel  aniarad  In  Blact  30,  II  OlllatanI  tram  Kapan) 


IS.  Supplementary  NOTES 


To  be  published  in  Biological  Cybernetics. 


IS  KEY  WORDS  (Canilnua  an  rararaa  alda  II  nacamaarf  and  Idanlllr  ay  Hack  r.uaibar) 

Neurons  Visual  Cortex 


Neurons  Visual  Cortex 

Generalization 

Specification 

Selectivity 

It  abstract  (Canilnua  an  raaaraa  alda  II  nacaaaary  and  Idantlly  by  black  numbar) 

M  A  rule  for  environmentally  dependent  modification  of  the  neuronal  state  is 
examined.  Under  the  rule,  the  neuron  selects  a  trigger  feature  that  matches 
either  a  particular  pattern  in  the  stimulus  set,  or  the  most  common  pattern 
component,  depending  on  a  certain  parameter.  Thus  a  neuron  may  evolve  to 
respond  to  its  stimulus  environment  in  one  of  two  capacities,  namely  speci¬ 
fication  or  generalization.,  Neurons  of  the  former  variety  are  labelled  "S-cel  s”; 
and  those  of  the  latter,  "G-cells"'.  In  the  model,  synaptic  modification  is 
modulated  by  two  postsynaptic  mechanisms,  which  act  antagonistically  to 


I  iAN  71 


COITION  or  I  NOV  ••  IS  OOSOLCTC 

S/N  0102-  LF.014-6601 


security  CLASSIFICATION  OF  THIS  PAGE  fBliA*  Oa»a  Kniaaad) 


02  19  066 


security  CLASSiriCATION  OF  THIS  RACE  rwhan  Data 


Strengthen  or  weaken  the  synaptic  connectivities.  The  functional  dependence 
of  these  mechanisms  on  the  postsynaptic  activity  is  shown  to  determine  whether 
the  neuron  acts  as  an  S— cell  or  a  G— cell.  A  circuit  is  proposed  for  a  module 
that  consists  of  a  G— cell  and  several  S— cells  sharing  a  common  set  of  inputs. 
By  inhibiting  the  G-cells,  the  S-cell  acts  as  a  contrast-enhancing  element, 
increasing  their  specificities  for  individual  patterns  in  the  stimulus  set. 

The  output  from  the  module  is  a  recorded  representation  of  the  environment 
with  respect  to  its  general  and  distinctive  features. 


A  Model  for  Generalization  and  Specification 
by  Single  Neurons  * 


Paul  W.  Munio  t 


Center  for  Neural  Science 
Brown  University 
Providence,  Rhode  Island  Q2912 


*  This  work  was  supporied  in  part  by  United  States  Office  of  Navii  Kesearcb  Contract  N00014-8I-K-0136. 
t  Cttireot  address:  Institute  for  Cognitive  Science  C-013  ;  UCSD  ;  La  Jolla,  CA  92093. 


Abstract.  A  nile  for  environmentally  dependent  modification  of  the  neuronal  state  is  examined.  Under 
the  rule,  the  neuron  selects  a  trigger  feature  that  matches  either  a  particular  pattern  in  the  stimulus  set, 
or  the  most  common  pattern  component,  depending  on  a  certain  parameter.  Thus  a  neuron  may 
evolve  to  re^ond  to  its  stimulus  environment  in  one  of  two  cq>acities,  namely  specification  or 
generalization.  Neurons  of  the  former  variety  are  labelled  *S-cells*;  and  those  of  the  latter,  *G-cells'.  In 
the  model,  synaptic  modification  is  modulated  by  two  postsynaptic  mechanisms,  which  act 
antagonistically  to  strengthen  or  weaken  the  syn^tic  connectivities.  The  functional  dependence  of 
these  mechanisms  on  the  postsynaptic  activity  is  shown  to  determine  whether  the  neuron  acts  as  an  S- 
cell  or  a  G-ccll.  A  circuit  is  proposed  for  a  module  that  consists  of  a  G-cell  and  several  S-cells  sharing 
a  common  set  of  inputs.  By  inhibiting  the  G-cells,  the  S-cell  acts  as  a  contrast-enhancing  element, 
increasing  their  specificities  for  individual  patterns  in  the  stimulus  set.  The  output  from  the  module  is 
a  recoded  representation  of  the  environment  with  respect  to  its  general  and  distinctive  features. 


1.  Introdoctifln 


1.1.  Trifser  featorei 

Barlow  (1972)  expresses  the  importance  of  single  neuron  response  characteristics  as  per¬ 
ceptual  substrates.  He  stresses  the  role  of  a  sensory  neuron’s  'trigger  features'  which,  according 
to  that  paper’s  third  proposition,*  'are  matched  to  redundant  patterns  of  stimulation  by  experi¬ 
ence  as  well  as  by  developmental  processes.'  The  present  paper  is  concerned  with  the  following 
question:  To  which  of  the  redundant  patterns  in  a  neuron’s  stimulus  environmat  is  its  trigger 
feature(s)  matched,  and  how  (in  a  mathematical  sense)  can  this  match  evolve  with  e:q>erience? 

Several  theories  describe  the  evolution  of  neuronal  trigger  features  (e.g.  von  der  Malsburg, 
1973  ;  Perez  et  al.,  1975  ;  Grossberg,  1976  ;  Amari  and  Takeuchi,  1978).  In  the  present  paper,  a 
scheme  for  neuronal  plasticity  is  put  forward  in  which  neurons  dynamically  adapt  to  their  indi¬ 
vidual  stimulus  environments  such  that  some  tune  to  particular  patterns  (specification)  while 
others  tune  to  the  most  prevalent  pattern  component  (generalization).  The  model  extends  the 
theory  of  Bienenstock  et  ai.  (1982)  for  development  of  orientation  selectivity  in  visual  cortex  to 
yield  either  maximization  or  minimization  of  selectivity,  depending  on  the  relation  between 
two  antagonistic  mechanisms  that  postsynaptically  modulate  synaptic  plasticity.  If  the  func¬ 
tions  are  of  the  same  form,  this  relation  can  be  expressed  in  terms  of  a  (fixed)  neuronal  parame- 
'er.  By  developing  response  characteristics  to  have  low  selectivity,  the  neuron  generalizes  by 
latching  its  trigger  feature  to  the  component  most  common  among  the  stimuli  in  its  environ¬ 
ment,  that  component  being  a  redundant  pattern  of  stimulation  as  in  Barlow’s  third  dogma.  On 
th«  other  hand,  by  pursuing  a  highly  specific  response  function,  the  trigger  feature  evolves  to 
:.ch  a  particular  pattern,  while  ignoring  a  maximally  broad  range  of  the  remainder  of  the 
stimulus  set.  Thus  some  neurons  pursue  common  features  and  some  pursue  distinctive  features 
in  the  environment. 


*Bailow’t  paper  consitu  of  five  ipeculaitve  proposiiiooi  (dogmaa)  which  he  auppoiti  with  penuaave  argumentf 
and  aubatantial  experimenlal  data. 
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1 Phystoloelcal  evidence 

The  development  of  highly  selective  neurons  in  mammalian  visual  cortex  (particularly  in 
cats  and  monkeys)  has  been  extensively  studied  under  variously  restrictive  rearing  conditions 
and  has  been  shown  to  depend  critically  on  the  visual  environment  (for  a  comprehensive  review 
of  the  literature,  see  Mov^on  and  Van  Sluyters,  1981).  The  stimulus  parameters  over  which 
selectivity  has  been  shown  to  develop  have  generally  been  orientation  and  spatial  frequency. 
However  Gross  et  al.  (1972)  found  neurons  in  the  pre-striate  cortex  of  macaque  monkeys  tuned 
to  features  of  higher  specificity  such  as  hands  and  faces. 

Cortical  neurons  exhibiting  low  or  zero  selectivity  in  the  orientation  domain  have  also 
been  observed.  These  are  particularly  common  in  layer  4  of  monkey  striate  cortex  (Hubei  and 
Wiesel,  1977).  If  such  neurons  are  present  in  the  cat,  they  are  certainly  less  common.  Kelly 
and  van  Essen  (1974)  report  symmetric  receptive  fields  in  some  units  but  classify  them  as  geni¬ 
culate  afferents.  Palmer  and  Davis  (1981),  on  the  other  hand,  give  evidence  that  nonselective 
cortical  neurons  exist  in  cat  visual  cortex,  but  are  rare.^ 

t.  The  model 

This  section  includes  the  formal  description  of  the  model  and  some  analysis  of  its  struc¬ 
ture.  No  argument  is  presented  in  this  paper  to  motivate  the  mathematics.  Interested  readers 
Ad  consult  Cooper  et  al  (1979)  and  Bienenstock  et  al  (1982).  Vector  quantities  appear  in 
Ad  type  and  are  specified  by  superscripts.  Components  of  vectors  are  not  in  boldface  and  are 
identified  by  subscripts.  Let  d,{t)  label  the  i-th  stimulus  component,  a  measure  of  the  activity 
ot  the  t-th  afferent  at  time  r.  Corresponding  to  each  component  is  a  synapse  of  strength 
There  are  N  components  to  both  d  and  m,  hence  they  are  both  vectors  in  N- 
•pace,  or  synapse  space.  A  stimulus  environment  refers  to  a  probability  density  for  d  in  N-^ace. 


ihii  study  of  single  neuron  response  properties  in  corte  t.  13  units  from  ■  total  of  2S7  weie  found  not  to  be 
otientatioa-edective.  On  the  basis  of  their  response  properties,  9  were  thought  to  be  geniculocottical  (l.GN  Y-cell) 
afferents.  Palmer  and  Davis  conclude  that  the  remaining  four  are  nonselective  cortical  neurons. 
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2.1.  The  transfer  function 

Let  the  net  integrated  depolarization  x(r)  be  the  thresholded  linear  eiqjrcssion: 


x(t)  =  max 


0 , 2m, (/)</,(»)  I 


(1) 


The  response  frequency  c(x)  of  the  neuron  is  thought  to  be  increasing  in  x  and  linear  in  some 
region,  but  with  nonlinearities  for  x  low  (the  firing  threshold)  and  x  high  (maximum  firing 
rate).  Since  x  rather  than  e  is  used  in  the  learning  rule,  the  precise  form  of  c(x)  is  not  impor¬ 
tant.  The  input-output  relation  is  determined  by  the  synaptic  weights  m,,  hence  the  input 
activity  levels  d,  are  termed  presynaptic  and  the  output  activity  x  (or  c)  is  termed  postsynaptic. 


A  neuronal  variable,  ^(r).  is  involved  in  the  synaptic  plasticity  rule  and  will  be  defined 
later.  Together,  the  synapse  vector  m  and  the  value  q(t)  make  up  the  neuronal  state  (in,q),  an 
(N-i-l)-dimensional  quantity.  For  simplicity,  the  components  m,  function  as  'ideal  synapses' 
(Nass  and  Cooper,  197S)  capable  of  changing  sign.  A  treatment  is  (pven  in  the  discussion  (sec¬ 
tion  4.2.2)  in  which  the  signs  of  the  synaptic  weights  does  not  change. 


2J.  The  modiflcatlan  rule 

The  rule  for  neuronal  plasticity  describes  the  changes  in  m  and  q  as  time  derivatives: 


mr(f)  =  ^{x,q)df(t) 

(2a) 

q(t)  =  P4>(x,0)x(r)  ;  p>0 

(2b) 

where  «t»(x,q)  =  (T2(x)-qa,(x) 

(2c) 

The  postsynaptic*  modulatory  function  is  given  in  terms  of  q  and  two  continuous, 

monotone  increasing  functions  of  x:  o’j(x),  and  Oiix).  To  complete  the  rule,  the  following 
restrictions  are  imposed.  The  function  is  subject  to  the  condition  (h(0.4)=0  for  all  q>0. 
Consequently,  the  two  functions  O]  and  a 2  must  each  vanish  for  x=0.  Also,  for  any  fixed 
value  4>0,  oj  and  02  vo  to  be  such  that  there  exist  at  most  one  other  value  x>0  at  which 


Htule*  for  tynapttc  modiflcitioa  are  often  exprewed  at  the  prodact  of  a  function  of  the  presynaptic  activity 
and  a  function  of  the  posttynaptic  activity.  This  approach  dates  hack  to  Thorndike’s  (1913)  *Law  of  Vttf  and  Itebb’s 
(1949)  *Neurophysiological  Postulate',  but  has  not  been  experimentally  veiiBed  until  recently  (Levy  and  Steward, 
1979;  Rauscbecker  and  Singer,  1981;  Singer,  1982;  Singer  and  Rauscbecker,  1982). 
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^(x^)=0.  The  parameter  ^  specifies  a  rate  constant  for  4  relative  to  m.  In  the  simulations 
below,  p  has  been  set  to  unity. 

2J.1.  PwaUcl  modiricaUoB 

Consider  the  above  equation  (2a)  for  m  and  note  that  it  is  linear  in  the  stimulus  d(t). 
The  modification  to  m  induced  by  a  given  stimulus  is  thus  parallel  or  antiparailel  to  d.  Of  all 
possible  input  stimuli  of  a  given  magnitude,  the  response  to  the  one  in  the  direction  of  that 
stimulus  d  is  most  greatly  affected  by  the  change  in  m.  Therefore  parallel  modification  (linear¬ 
ity  of  m  in  d)  has  the  following  behavioral  implication  for  a  system:  letwmng  induced  by  a  par- 
ticular  stimulus  more  greatly  ittfluences  future  responses  to  that  stimulus  than  to  others  of  the  same 
magnitude. 

The  sign  of  the  function  determines  whether  the  synapses  get  stronger  or  weaker  -  i.e. 
whether  m  is  parallel  or  antiparallel  to  d.  For  a  given  value  of  q,^  has  at  most  two  zeros  in  x, 
one  of  them  being  x  =0.  Thus  the  sign  of  can  vary  over  x  in  four  ways,  depending  on  the 
value  of  q.  Two  of  these  are  illustrated  in  Figure  la  in  which  for  q  sufficiently  small  or  large 
and  x>  0,  is  respectively  positive  or  negative  definite.  For  intermediate  values  of  9,  ^  is  first 
positive  and  then  negative  (Figure  lb)  or  vice  versa  (Figure  Ic).  These  last  two  situations  can 
be  labelled  by  the  sign  of  the  partial  derivative  at  the  point  x  =0(9)>0  where  ^(0(9),9)=0. 
That  sign  depends  strictly  on  the  forms  of  <ri(x)  and  ojfx)  and  hence  only  one  of  these  two 
cases  is  possible  for  a  given  neuron. 

2.2 Restrictions  on  a  1  and  02 

As  shown  in  Figure  1,  cross  sections  of  <i>(x,9)  at  constant  q  can  evolve  in  just  two  ways 
as  q  increases  from  0.  For  q  less  than  or  equal  to  a  certain  value  9^,  ^  is  always  positive  for 
X>0,  and  for  929*,  ^  is  always  negative.  Thus  a  function  0(9)  can  be  defined  on  the  interval 
qi.{q,Ak)  such  that  x=e(9)>0  is  the  second  zero  of  ^  -  i.e.  ^(0(9),9)=0.  The  inverse  func¬ 
tion  9=0"’(x)  is  given  by: 


The  following  theorem  establishes  a  condition  on  Oi  and  1T2  allows  <)>(x^)  at  most  one 
zero  on  the  domain  x  >  0  for  any  fixed  value  of  4 . 

Thecrtm.  For  any  6xed  positive  value  of  f ,  the  fuaction  ^hete  a^{x)  and  ir2(')  ■re  poative 

and  diffetentiable  on  the  domain  a>0,  has  at  most  one  zero  for  z>0  if  the  Wronskian  W(iri(z)^2('))  '*  cither 
always  positive  or  always  negative  for  x>Q. 


Proef.  For  a  6xed  positive  value  of  f ,  define  a  set  of  values  x,  on  the  domain  x>0  such  that  ^(xj4)N)  for  all  i.  If 
the  sign  of  the  partial  derivative  (x  -x, )  is  constant  for  all  values  of  x  >  0,  then  there  can  only  be  one  zero  of  d  in 


that  domain.  Note  that  9-a2(xj)/(ri(x,')  for  all  i.  ^,(x  -x,)  is  shown  to  be  equal  to 


.l>,(x-e(?))  =  02'  ■■  “■•'i' 
<^1 

J  1"'  "’1 

Vi  KTl'  <t2'l 


(«) 


=  W(<r  1.9  i)/a  I 

Kence  the  sign  of  d^(x  ~9(q))  is  the  same  as  the  sign  of  W  (x).  This  completes  the  proof. 


The  (constant)  sign  of  W(j:)  will  be  shown  to  determine  the  feature  abstracting  property 
of  the  neuron.  If  W(x)  is  always  positive,  the  neuron  generalizes  (a  'C-ccll')  and  if  it  is  nega¬ 
tive  the  neuron  tunes  to  a  specific  stimulus  (an  "S-ceir)-  This  is  demonstrated  in  Section  3. 
The  sign  of  0'(9)  is  constant  for  all  values  of  <7  €  ,q„)  and  is  also  the  same  as  those  of  W (x) 

and  tt»j,(x=0(9)).  This  is  because  the  inverse  function  0~'(x)  is  also  monotone  increasing  or 
decreasing: 

dO"‘(x)  <ri<r2'-<yr«^2  W  d'x(®.^) 

dx  cr?  af  <ri 

In  addition  to  the  signs  of  rbxi  and  9\q),  there  is  a  fourth  quantity  providing  an 

equivalent  condition.  Those  three  quantities  are  positive  if  and  only  if  the  logarithmic 


derivative  (LD)  of  ai(x)  is  gt^ter  than  the  LD  of  <r^jc)  for  all  x>0,  and  negtdive  if  and  only 
if  the  reverse  condition  holds.  Thus  the  LD  of  one  function  is  required  to  be  consistently 
higiier  than  the  other.  The  feature  abstracting  property  of  the  neuron  depends  only  on  which 
LD  is  greater. 

2JJ.  Some  allowed  functions 

Now  consider  functions  ai  and  02  c^n  be  expressed  as  a  common  function,  only 
with  different  scale  parameters: 

(r,(x)  =  (6) 

The  sign  of  the  Wronskian  is  now  related  to  a  functional  of  a(x)  by  the  following  corollary. 
Note  that  the  amplitude  parameters  and  p.2  are  not  important  here  since  neither  the  sign  of 
the  Wronskian  nor  the  logarithmic  derivatives  depend  on  them. 

Corollary.  The  fuoctioa  ((>(jr  ,v)=  <f(x/ij2)-?£r(x/'ni)  has  at  moat  one  zero  forx>0if  t[(r(x)]>0for  allz>0,  where 
“(*)"»(* )«'(■')•  W  this  cooditioo  holds,  tbeo  the  agn  of  the  Wionskiao 
W((T(x/n,).o(x/ri2))  equals  sgnlqj-ni). 

Proof.  If  the  LD  of  olx/t))  is  monotone  increasing  in  t|,  then  the  LD  for  the  sigma  function  with  a  higher  value  of 
i|  will  always  be  greater.  Thus  a  derivative  is  taken  with  respect  to  v|  and  the  sign  of  the  result  is  seen  to  be  equal  to 
the  sign  of  /,[(r(x)]: 

d  I  g  '(s  /n)  ,  fyv 

dq  [i)<r(x/q}J  (iiv(>))^  '' 

where  r  ■  — 

•q 

The  case  for  negative  definite  L(x)  (for  which  the  signs  are  opposite)  con  be  ruled  out  by  considering  the  behavior  of 
.4i(x .  '  for  f  ’’wjpx  io  the  neighborhood  of  x  In  this  case,  4  has  the  same  sign  as  o  (x  /q2)~o  (x  A)]).  Hence,  as  x 
increases  from  zero,  ^  takes  on  the  sign  opposite  to  vi2~')i-  ^<’1*  tbat  for  0<x <•(«),  the  sign  of  4  must  be  oppo* 
site  to  that  of  the  Wronskian.  Therefore  only  the  case  in  which  the  LD  increases  with  q  applies,  so  the  condition 
L(x)>  0  is  sufficient  and  the  signs  of  the  Wronskian  ete.  are  equal  to  sgn('q2~qi)- 


Certain  functions  used  to  approximate  the  relation  between  net  somatic  depolarization 
and  neuronal  firing  rate  (i.e.  c(x))  satisfy  this  corollary.  Among  such  functions  are  ln(l+x) 
(Agin,  1964)  and  sigmoidal  functions  of  the  form; 


o(x)= 


x'+l 


pal 


(8) 


which  approximate  typical  bounded  threshold  functions  like  that  measured  by  Chapman  (1966) 
in  the  crab  and  by  Creutzfelt  et  al.  (1970)  in  cat  retinal  ganglion  cells.  Hence  either  oi  or  03 
might  be  related  to  the  firing  rate  c . 

For  the  case  where  <ti  and  a 2  ^te  of  the  same  form  cr(x/T)),  the  sign  of  the  VVronskian  is 
seen  to  depend  on  which  function  has  the  greater  scale  factor  t)  for  the  argument.  Thus 
sgn(lV(x))=sgn(T)2-T)j),  and  so  for  t|2>tii  the  neuron  tunes  to  a  particular  stimulus  in  its 
stimulus  environment,  and  for  t)2<  vii  the  neuron  is  driven  towards  generalization. 


23.  Stability 


Trajectories  arc  drawn  in  the  m  -q  plane  for  a  one-synapse  neuron  receiving  a  constant 
stimulus  do-  1  (Figure  2).  It  can  be  seen  that  the  neuron  is  always  driven  to  a  finite  final  state. 
Note  that  if  the  initial  value  of  q  is  too  large  the  final  value  of  m  will  be  zero  (the  neuron  loses 
all  responsivity).^  Tlie  final  states  of  the  system  are  not  perturbed  by  small  amounts  of  signal 
noise  on  the  average.  This  is  because  the  expected  value  of  (m  ,9)  is  along  a  trajectory.  Con¬ 
sider  noise  uniform  on  the  interval  [-a,aj: 
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Hence  perturbations  from  equilibrium  states  are,  on  the  average,  driven  back  to  the  same 
equilibrium  point  in  the  m-q  plane. 


^eoce  it  may  be  advantageoui  to  asnime  that  It  ihoaid  be  noted  however  that  this  condition  is 

not  lufficient  for  multi-pattern  environments.  Simulations  indicate  that  the  initial  values  of  the  synaptic  weight* 
be  large  in  order  to  guarantee  a  nontrivial  final  state. 
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3.  Feature  abstraction 

The  dependence  of  a  neuron's  feature  abstraction  properties  (specification  or  generaliza¬ 
tion)  on  the  sign  of  the  Wronskian  has  already  been  mentioned.  The  degree  of  selectivity  and 
the  quality  of  feature  abstraction  depend  critically  upon  the  structure  of  the  stimulus  environ¬ 
ment  in  N-space.  Therefore  selectivity  is  precisely  (albeit  somewhat  arbitrarily)  defined  and  dis¬ 
cussed  in  this  section  for  a  variety  of  pattern  sets  and  the  feature  abstraction  properties  of  the 
model  are  analyzed. 

3.1.  Stimulus  envlronmeiits  and  selectivity 

It  is  important  to  emphasize  the  role  of  the  neuron’s  stimulus  environment  as  a  context 
for  both  the  development  and  testing  of  neuronal  response  characteristics.  Since  each  stimulus 
can  be  represented  in  the  same  N-dimcnsional  space  as  the  synaptic  state  m,  a  stimulus  environ¬ 
ment  can  be  thought  of  as  a  probability  density  function  on  that  space.  Under  a  given  learning 
rule,  the  locus  of  stable  (final)  states  m(/-  ®  )  is  determined  by  the  stimulus  environment. 

Neuronal  selectivity  is  measured  with  respect  to  a  test  environment,  a  set  of  stimuli  which 
one  must  be  careful  to  distinguish  from  the  learning  environmetu.  While  the  neuronal  state  may 
undergo  a  measurable  (adaptive)  change  under  the  influence  of  the  learning  environment 
(indeed  this  is  often  the  whole  point),  the  test  environment  is  used  to  perform  a  measure  on 
the  state.  Even  so,  in  the  analysis  of  this  model  the  learning  environment  is  virtually  identical 
(except  for  the  addition  of  a  random  noise  term)  to  the  test  environment.  Also,  the  inevitable 
influence  of  the  measurement  on  the  state  itsdf  is  idealized  to  be  nil. 

A  neuronal  response  function  c(m,£)  can  be  calculated  over  any  pattern  set  E  for  a 
given  synaptic  state  m.  A  piccisc  definition  of  selectivity  tor  a  general  stimulus  density  func¬ 
tion  P  {dE)  was  given  by  Bicncnstock  (1980).  This  can  be  paraphrased: 

fc(m,E)P(dE) 

S  (mji  )  =  1  -  — - -  1  - 

t-tOMX 

vhere  Caua  -  nyi*(c(m.£)) 


(10) 


Note  that  S  (m^  )  C  [0,1)  with  zero  indicating  a  unifonn  response  across  E  and  unity  being  an 
idealized  (zero-width)  response  peak.  The  function  is  illustrated  (Figure  3)  for  a  stimulus 
environment  that  is  characterized  in  terms  of  a  single  parameter. 

For  a  given  environment  E ,  the  selectivity  S  (ni,E)  is  a  scalar  function  of  the  state  m.  In 
Figure  4,  lines  of  constant  selectivity  are  plotted  for  an  environment  consisting  of  two  discrete 
patterns  d‘  and  d^.  Restrictions  on  the  synapses,  such  as  the  sign  of  m,,  may  prevent  a  cell 
from  realizing  its  optimal  state.  In  such  cases,  simulations  have  indicated  that  the  neuronal 
state  is  driven  to  the  limiting  boundary. 

3.2.  Discrete  pattern  sets 

3.2.1.  Two  patterns  In  two  dimensions 

This  is  the  simplest  case  in  which  the  feature  abstraction  properties  of  the  model  can  be 
analyzed.  The  neuronal  state  space  is  illustrated  in  Figures  Sa  and  Sb.  For  each  pattern  there 
exist  two  surfaces  at  which  <h=0.  One  surface  is  the  plane  m*d=0  and  the  other  surface 
represents  the  relation  m*d-0(f ).  At  the  intenections  of  the  <^(m*d*,9)=0  surfaces  and  the 
<)>(m*d^,9)=0  surfaces  are  equilibrium  states  where  for  both  patterns. 

There  are  four  distinct  loci  of  equilibrium  states.  One  consists  of  the  q-axis  (m  =0),  two 
represent  maximally  selective  states  (one  for  each  pattern),  and  one  is  made  up  of  states  having 
minimum  (zero)  selectivity.  In  the  case  where  the  Wronskian  is  positive,  only  points  belonging 
‘o  the  maximally  selective  equilibrium  locus  are  stable  and  if  the  Wronskian  is  negative  only 
the  minimally  selective  (nontrivial)  equilibrium  points  is  stable. 

For  a  given  synaptic  vector  m,  is  positive  for  9<0”’(m’d’)  and  ?<0~*(m'd^  -  i.e. 
below  both  surfaces  m’d'  =0(9)  (cf  Figure  2).  Above  the  two  surfaces,  ^  is  negative.  Hence  in 
»a?h^;]case  the  neuronal  state  is  driven  toward  those  surfaces  and  eventually  ends  up  between 
tuem,  at  their  intersection  (zero  selectivity),  or  on  the  9 -axis  (m  =  0).  These  cases  are  con* 


Caw  1:  6~'(in*d')>9>0~‘(iii‘d^.  The  argument  for  this  case  can  also  be  applied  to  the 
case  where  d‘  and  d^  are  reversed.  Here  ^(m*d’)>  0  and  ^(■■i*d^<  0,  so  the  response  tends 
to  increase  and  the  response  m*d^  tends  to  decrease.  This  drives  the  state  toward  minimum 
selecti>dty  (i.e.  toward  m‘d^=m*d^  if  m*d*<m’d^  and  toward  maximum  selectivity  (in  this  case 
toward  m*d^=0)  if  m*d'>m’d^.  Therefore  the  feature  abstraction  property  of  the  neuron 
depends  on  the  sign  of  OXf)  hence  on  the  signs  of  W  (jc)  and  =0(9))  as  shown  earlier 
in  (5)). 

Case  2:  m'd‘=m‘d^=0(9).  In  this  case  the  neuron  is  in  a  minimum  selectivity  equilibrium 
state  and  so  the  stability  of  the  state  must  be  considered.  Any  noise  in  the  stimulus,  response, 
or  plasticity  function  will  perturb  the  state  from  equilibrium.  By  the  above  arguments  it  is 
seen  that  an  S-cell  will  be  driven  away  from  this  locus  and  a  G-cell  back  toward  it. 

Ctae  3:  m‘d’=m*d^=0  (the  ^-axis).  The  ^-axis  is  a  locus  of  trivial  (zero  response  to  all 
stimuli)  states  which  is  stable  for  However  it  is  never  reached  (again  see  Figure 

2)  if  the  initial  value  of  ^  is  small  (q(r  =0)  «  0~‘(x  =0)). 

m-plane  projections  of  numerically  evaluated  trajectories  are  shown  in  Figure  6  for  2- 
pattem  environments. 

3J.2.  Linear  Independence 

A  linearly  independent  set  of  AT  stimulus  patterns  presented  to  a  neuron  stochastically  is 
treated  as  an  ideal  case,  since  the  model  can  function  perfectly  in  this  situation.  If  each  pat¬ 
tern  is  presented  with  non-zero  probability,  then  neurons  of  both  types  will  asymptotically 
approach  their  optimal  selcctivitics:  zero  for  G-cells  and  1-1/X  (the  maximum)  for  S-cells. 
While  this  statement  has  not  been  proven,  it  is  strongly  supported  by  computer  simulation. 
These  ideal  final  states  are  possible  for  a  linearly  independent  stimulus  environment  and  onfy 
far  such  an  environment.  This  can  be  seen  by  considering  the  syn:q>se  space.  The  conditions 
ni*d=0  for  JC  -1  patterns  and  m*d>0  for  the  last  one  (maximum  selectivity)  can  only  be  simul¬ 
taneously  satisfied  only  if  all  AT  patterns  are  independent.  Similarly,  linear  independence  is 
required  to  reach  zero  selectivity  nontrivially  -  i.e.  to  satisfy  the  condition  that  in*d  be 
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constant  for  all  environmental  stimuli  d. 

An  important  feature  of  the  model  can  be  seen  in  such  an  environment  (and  proven  for  2 
patterns),  namely  that  the  neurons  (asymptotically)  acheive  optimal  selectivity  even  if  the  pat¬ 
terns  are  not  presented  with  uniform  probability.  Hiis  is  perhaps  more  impressive  with  respect  to 
the  generalizing  neurons.  A  scheme  whereby  the  synt^ses  nmply  average  the  stimuli  over  time 
will  extract  the  most  common  feature  components  according  to  the  probability  distribution. 
However,  in  this  model  the  most  common  feature  component  anor^  the  individual  stimuli  is 
extracted. 

3JJ.  Linear  dependence 

In  general,  the  above  'idear  final  states  cannot  be  do  not  exist  for  an  environment  in 
which  the  stimuli  are  not  independent.  Nevertheless,  both  cell  types  are  driven  toward  optimal 
states  under  this  condition.  For  S-cells,  a  high  selectivity  is  attained  except  under  overwhelm¬ 
ing  circumstances,  such  as  the  situation  in  which  all  stimulus  patterns  differ  only  in  their  mag¬ 
nitude.^  If  all  stimulus  components  are  assumed  to  be  positive,  it  is  even  difficult  to  design  an 
environment  under  which  the  G-cell  cannot  reach  a  low-sdectivity  state.  This  is  because  for 
patterns  having  little  in  common  (and  therefore  not  very  *generalizable*),  all  the  synapses  can 
simply  grow  very  large  so  that  just  about  any  stimulus  evokes  a  high  response.  The  principal 
danger  in  this  situation  is  overgeneralization:  ideally,  a  G-ceO  will  only  respond  with  low  selec¬ 
tivity  as  measured  across  its  stimulus  environment. 

3J.  Continuous  pattern  sets 

Certainly  not  all  stimulus  sets  are  discrete.  For  example,  physical  stimulus  feature  dimen¬ 
sions  are  almost  ail  continuous  since  they  are  intensity  measures.  In  this  section  continuous 
stimulus  environments  that  have  a  periodic  dependence  on  one  or  two  parameters  are  dis- 

^nch  patteni  teti  can  be  (bought  of  as  a  liagle  patlero,  ia  (hat  (be  iofonoationai  content  of  a  atinulua  ia 
eontaioed  in  the  retalvt  vetues  of  the  componeota.  Thus  the  diiection  of  a  vector  (atinulua  or  aynaptic) 
ia  aaaociated  with  information  and  the  magnitude  with  iotenaity.  Gati  and  Tveraky  (1982)  relate  tbew  to 
the  'qualitative*  and  'quantitativeT  aapecta  of  a  atinulua.  Note  that  to  incorporate  thia  notion  into  the 
present  model  would  requite  that  the  operant  deOnition  of  selectivity  he  modided. 


cussed.  This  is  done  as  an  approximation  to  a  visual  environment  consisting  of  oriented  con¬ 
trast  edges  where  stimulus  orientation  is  the  varying  parameter.  Because  they  have  been  widely 
used  to  measure  the  selectivity  properties  of  neurons  in  riate  cortex,  such  environments  are 
useful  for  comparison  of  the  modd  with  experiment. 

3.3.1.  Periodicity  in  one  parameter 

The  behavior  of  the  model  in  an  environment  varying  periodically  in  a  single  parameter 
provides  a  gpod  illustration  of  how  the  model  functions.  Such  environments  map  onto  closed 
loops  in  N-space.  Here  these  are  assumed  to  be  'reasonably*  convex  so  that  if  two  patterns  are 
close  in  the  stimulus  space,  then  they  are  also  close  in  terms  of  the  environmental  parameter 
(the  converse  holds  for  a  continuous  environment  by  definition).  The  prindple  of  patalld 
modification  operates  in  such  an  environment  such  that  as  the  various  patterns  stimulate  the 
cell  they  have  the  following  cumulative  effect.  The  selectivity-maximization  property  of  the  S- 
cell  functions  by  'encouragingf  patterns  that  evoke  responses  greater  than  0(f)  and  'discourag¬ 
ing  patterns  that  evoke  smaller  responses  (upper  part  of  Figure  7).  As  f  varies  according  to 
(2),  the  threshold  and  the  response  converge  such  that  some  responses  lie  above  the  threshold 
and  others  below.  The  neuronal  response  curve  then  becomes  'separated*  (i.e.  the  difference 
between  the  minimum  and  maximum  response  increases)  about  the  threshold  0(f).  In  the 
-iwer  part  of  Figure  7,  the  reverse  process  is  seen  to  drive  the  response  curve  toward  low  selec- 
ivity. 

33J.  Periodicity  In  two  parameters 

A  pattern  set  periodic  in  two  independent  parameters  has  a  more  complex  structure.  The 
environmental  topology  progresses  from  a  closed  loop  to  a  torus  with  the  dependence  on  an 
additional  parameter.  The  behavior  of  the  model  has  not  been  treated  analytically,  but  nonri¬ 
gorous  results  have  been  obtained  by  numerical  methods  (Figure  8).  These  simulations  indicate 
that  the  response  surface  of  a  G-cell  becomes  flat  and  that  of  an  S-cdl  becomes  peaked  -  i.e. 
the  cell  types  respectively  seek  minimum  and  maximum  selectivity  over  both  parameters. 


4.  Dlscnsslmi 


4.1.  4  as  a  time  average  over  x(t) 

This  model  is  an  extension  of  the  Bienenstock  et  al.  (1982)  theory  for  selectivity  maximi¬ 
zation.  In  that  theory  the  function  ^  is  postulated  to  be  a  function  of  the  instantaneous 
response  c  and  the  mean  response  c .  For  a  given  value  of  c .  ^  is  negative  for  c  below  a  cer¬ 
tain  threshold  value  0  and  positive  for  e  >  0,  where  0  depends  on  c .  The  syston  is  stable  for 
functions  fi(c)  that  increase  in  a  'faster-than-lineai'  fashion  with  c  (e.g.  6(c)=^  ;p>  1). 

A  more  direct  extension  of  that  theory  to  selectivity  minimization  is  possible  (Munro, 
1983),  by  inverting  the  requirements  on  the  sign  of  ^  and  on  the  form  of  0  so  that  it  is 
'slower-than-linear*  in  c  (ie.  p<l).  Hence  the  important  mathematical  conditions  on  the 
model  appear  to  be  that  change  sign  once  as  a  function  of  the  response  and  that  the  value  0 
vary  temporally  with  the  response  according  to  a  function  which  must  satisfy  certain  condi¬ 
tions.  The  feature  abstracting  property  of  a  given  neuron  then  relies  on  the  signs  of  two 
parameters,  namely  <^,(c=0(c))  and  p-1.  Of  the  four  possible  combinations,  one  gives 
specification,  one  gives  generalization,  and  the  others  do  not  converge  to  stable  final  states. 
Hence  the  following  condition  is  imperative:  sgn(4>c(c  'Hie  interdepen¬ 

dence  of  these  two  parameters  is  not  so  artificial  in  the  ^proach  taken  by  the  present  paper. 

It  has  been  shown  that  sod  9  (4)  ate  related  (5).  The  success  of  the  model 

and  its  similarity  to  the  theory  of  Bienenstock  et  al.  depend  on  q(t)  playing  the  role  of  c .  By 
integrating  (2b),  one  can  see  that  q,  like  c,  is  a  time  average  of  a  measure  of  the  postsynaptic 
actiA'ity: 

f  f 

q(t)  =  qge  +  p/e  '  '  ' 

0 

Note  that  the  first  term  decays  with  time. 


4^  Visual  cortex 


4J.1.  Simalatloa  of  experiments 

Since  this  model  is  a  direct  descendant  of  the  theory  of  Bienenstock  et  al.  (1982)  for 
selectivity  maximization  in  visual  cortex  (see  Section  4.1),  it  comes  as  no  surprise  that  the 
scheme  for  the  S-cell  explains  the  same  body  of  physiological  data.  Computer  simulation  indi¬ 
cates  that  the  two  theories  behave  nearly  identically,  from  a  qualitative  standpoint,  in  several 
variously  manipulated  visual  environments  (e.g.  normal  rearing,  monocular  and  binocular 
deprivation,  and  reverse  suture).  However  results  differ  for  simulation  of  the  artificial  stra¬ 
bismus  paradigm.  A  toroidal  environment  (section  3.3.2)  is  used  to  describe  strabismic  input. 
Recall  that  the  S-cell  tunes  independently  with  respect  to  both  parameters  -  i.e.  it  develops 
high  specificity  in  both  eyes,  but  not  necessarily  to  a  common  pattern.  In  the  Bienenstock  et 
al.  formulation,  such  an  environment  drives  a  neuron  to  a  monocular  state  (Munro,  1983). 
That  is,  tuning  is  accomplished  with  respect  to  one  of  the  two  parameters.  These  results  are 
purely  empirical  and  have  not  been  derived,  so  it  is  conceivable  that  the  behavior  of  either  or 
both  models  in  a  noncorrclatcd  binocular  pattern  space  depends  on  some  parameter(s).  Data 
from  Hubei  and  Wiesci  (196S)  suggest  that  strabismic  rearing  results  in  fewer  binocular  cells, 
and  thus  supports  the  scheme  of  Bienenstock  et  al.  as  a  model  for  visual  cortex  neurons.  On 
the  other  hand,  Blakemore  and  Van  Sluyters  (1974)  report  the  existence  of  binocular  neurons 
with  a  high  orientation  disparity  between  the  two  eyes. 

4J.2.  Contrast  enhancement 

It  is  generally  accepted  that  the  projection  from  the  principal  visual  input  (the  lateral  gen¬ 
iculate  nucleus  of  the  thalamus)  to  striate  cortex  is  overwhelmingly,  and  perhaps  exclusively, 
excitatory.  Inhibitory  influence  is  therefore  thou^t  to  be  mediated  by  cortical  intemeurons. 
Figure  9  presents  a  scheme  which  is  in  accordance  with  these  observations.  A  set  of  neig)ibor- 
ing  neurons  is  shown  that  share  a  common  set  of  excitatory  afferent  fibers.  A  variety  of  S-cells 
serve  to  tune  to  a  corresponding  selection  of  environmental  stimuli,  whereas  fewer  G-cells  are 


required  (perhaps  only  one). 


By  inhibiting  all  the  S-cells  in  this  'cluster',  the  G-cell  can  increase  their  selectivitics. 
Inhibition  by  a  G-cell  raises  the  excitation  threshold  of  an  S-cell  uniformly  in  as  much  as  the 
response  of  the  G-cell  is  uniform  across  the  environment.  A  similar  circuit  has  been  proposed 
by  Grossberg  (1976).  He  proposes  a  scheme  in  which  the  inhibitory  intemeuron  receives  inputs 
from  the  surrounding  feature  detectors.  The  resulting  re^onse  is  of  minimal  selectivity  to  the 
extent  that  the  input  weights  are  equal. 

Secondary  evidence  for  such  a  circuit  is  found  in  the  visual  cortex  of  macaque.  Hubd 
and  Livingstone  (1%2)  and  Tootell  et  al.  (1982)  have  found  periodic  clusters  of  non-selective 
cells  centered  in  ocular  dominance  columns.  Also,  Hendriclrson  et  al.  (1981)  report  a  similar 
distribution  for  the  enzyme  glutamic  acid  decarboxylase  (GAD)  which  synthesizes  the  inhibi¬ 
tory  neurotransmitter  GABA.  They  have  found  localized  GAD-rich  areas  that  run  parallel  to 
ocular  dominance  columns.  An  examination  of  the  fine  structure  of  the  axon  terminals  of 
low-selectivity  neurons  would  directly  test  the  hypothesis  that  they  are  inhibitory. 

4J.  System  organlzatioo 

A  conjecture  can  now  be  made  explicit  in  terms  of  four  neuronal  types,  defined  according 
to  their  optimal  selectivitics  (minimum  or  maximum)  and  their  influence  on  other  neurons 
(excitatory  or  inhibitory),  namely  that  a  circuit  can  be  constructed  from  these  four  neuronal 
types  (S^,S',G^,G')  that  will  evolve  to  classify  any  stimulus  set  in  terms  of  similarities  and 
differences.  That  is,  such  a  neural  network  might  sclf-or^nize  such  that  a  taxonomic  represen¬ 
tation  of  the  environment  is  constructed.  This  hypothetical  network  is  not  easily  realized. 
The  problem  is  to  discover  an  architecture  that  provides  an  appropriate  context  for  each  neu¬ 
ron.  Since  this  problem  has  not  been  solved,  let  us  consider  how  it  might  be  appraarhed. 


An  initial  step  is  to  investigate  the  properties  of  small  local  neuron  circuits  or  modules  as 
^.yeuased  by  several  authors  including  Mountcastle  (1978)  and  Szentagothai  (1975).  The  con¬ 
tra,!  enhancement  scheme  in  Figure  9  is  a  candidate  for  such  a  functional  unit.  The  problem 
of  these  modules  might  be  arranged  to  form  larger  components  must  also  be  eventually 


addressed.  At  each  level  the  circuits  are  presumably  genetically  coded  stroctures,  within  which 
individual  neurons  adapt  to  informational  aspects  of  their  respective  stimulus  environments. 

Consider  a  system  organized  in  a  hierarchical  fashion,  reflecting  progressively  complex  per¬ 
ceptual  stages  (e.g.  sensation  -  recognition  •  association  •  cognition  ...).  It  would  be  a  mistake 
to  assume  that  these  stages  lie  in  a  strictly  serial  path  since  system  processing  is  certainly  paral¬ 
lel  to  a  certain  degree.  The  combined  hierarchical/paralld  flow  of  information  in  the  visual  sys¬ 
tem  has  been  recently  examined  by  Van  Essen  and  Maunsdl  (1983)  in  terms  of  so-called  'func¬ 
tional  streams'. 

The  nature  of  the  stimulus  environments  at  successive  stages  of  neural  processing  prob¬ 
ably  becomes  increaangly  discrete.  Shepard  and  Podgomy  (197S)  point  out  that  symbolic 
stimuli  are  discretely  coded  while  physical  (nonsymbolic)  stimuli  vary  continously  along  one  or 
several  dimensions:  'For,  whereas  we  can  continuously  shift  a  color  (for  example,  blue)  in 
brightness,  hue,  and  saturation  until  it  becomes  as  similar  as  we  wish  to  any  other  color  (for 
example,  green),  we  cannot  continuously  deform  a  word  ’blue*  to  another  word  ’green'  without 
passing  through  intermediate  configurations  that  are  not  words  at  all.'  Thus  it  is  expected  that 
inputs  to  the  higher  (more  'gnostic')  stages  are  more  distinctly  separated  than  early  (sensory) 
input  patterns.  Inhibition  is  a  powerful  tool  for  separating  patterns  (see  section  4.2.2)  and  as 
such  may  play  an  important  role  in  the  formation  of  discrete  (symbolic)  representations. 
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Figure  Ceptions 


Figure  I.  Functions  qfTi(x),  <T2(x),  and  ^(x^)  are  plotted  against  x  for  two  values  of  q  (qi<q^  in 
three  situations,  qtri  and  02  are  shown  above  and  ^  is  shown  below,  a  qi<q^  ;  92^ 9»-  ^  Both  values 
of  q  in  the  interval  q^(qa^b)  where  the  Wronskian  1^(01,02)  <  0.  c  The  same  for 

IF  (<r  1,02)  >  0.  (See  theorem  in  text).  Note  thtf  the  sequence  of  zero  crossing  for  and 

^2)  c  >>  opposite  to  that  in  b. 

Figure  2.  Parabolic  trajectories  given  by  dqfdm  ~  pm,  a  simulation  of  a  one-synapse  neuron  receiving 
constant  input  d^  =  1.  Stable  equilibrium  states  are  indicated  by*.  Note  that  states  with  q(t=0)<qa 
are  driven  to  nontrivial  (m  #  0)  final  states.  Functions  cr,  of  the  form  (6)  were  used  for  the  two  cases; 
■  'n2>  'Hi  and  b  y\2<  th. 

Figure  3.  This  response  curve  illustrates  the  definition  of  selectivity  given  by  (10)  for  a  one-parameter 
(«)  stimulus  environment.  The  domain  of  <■>  together  with  the  response  range  (zero  to  the  maximum 
M)  determine  a  rectangle.  The  fraction  of  this  rectangle  which  docs  not  lie  below  the  response  curve  is 
the  selectivity  cf  the  response.  (From  Bicncnstock  ct  al.,  1982,  with  permission) 

Figure  4.  Lines  of  constant  selectivity  for  two  2-pattcm  environments,  which  have  a  maximum 
selectivity  ul  0.5  (l-l/X  for  X  patterns).  Note  that  there  is  a  line  of  maximum  s.Icctivity  orthogonal 
to  each  pattern  and  that  the  line  of  zero  selectivity  is  orthogonal  to  the  difference  between  the 
patterns.  «  if  the  two  patterns  have  a  common  norm,  the  locus  of  zero-selectivity  lies  midway  between 
the  patterns,  b  if  they  have  different  norms,  the  isoselectivity  contours  are  skewed  toward  the  weaker 


stimulus. 


I 


Figure  5.  Curved  surfaces  give  loci  of  nontrivial  equilibria  with  respect  to  each  of  the  stimuli  in  the  2- 
pattem  environment  d'=(l,0),  d^=(0,l).  Each  surface  is  defined  by  translating  the  curved  lines  in 
Figure  2,  in  the  direction  orthogonal  to  the  planes  in  which  they  lie.  Assuming  7(f  =0)  is  sufficiently 
small,  the  neuronal  state  converges  to  one  of  the  following  Icei,  each  defined  as  the  intersection  of  a 
<})(m*d'=0)  surface  and  a  4>(m‘d*=0)  surface,  a  S-cell:  c;  or  C2.  contours  defined  by  maximally  selective 
states  satisfying  in"d=0  for  one  pattern  and  fl=0"'(m*d)  for  the  other,  b  G<ell;  cji 
(in'd*=ni'd*=0(^)),  a  contour  consisting  of  minimally  selective  states. 

Figure  6.  m-plane  projections  of  trajectories  arc  shown  for  the  two-pattem  environments  of  Figure  4: 
common  norm  -  a  S-cell:  W  >  0,  b  G-cell:  W  <  o  ;  different  norms  -  c  S-cc!l,  d  G<cll. 

Figure  7.  Behavior  of  the  model  is  shown  in  an  environment  that  is  periodic  in  one  parameter.  If  the 
response  generated  by  a  particular  stimulus  gives  a  positive  value  for  <}>,  then  the  resulting  synaptie 
potentiation  increases  the  response  function  over  a/J  patterns.  The  increase  is  a  maximum  for  patterns 
in  the  same  direction  as  the  stimulus.  This  illustmtcs  the  notion  of  parallel  modification  (section 
2.2.1).  With  sufficient  exposure  to  the  ensironment,  the  response  curve  thus  becomes  maximally 
selective  for  W  >0  (upper  figure)  or  minimally  selective  for  W  <0  (lower  figure). 

Figure  8.  An  environment  that  varies  periodically  over  two  independent  parameters  drives  a  initial 
random  state  having  the  response  surface  shown  in  a  to  the  final  state  b  for  IV  >0.  and  to  c  tor  V/  <0, 
The  stimulus  set  consisted  of  400  patterns  in  a  20^imcnsional  .space. 

Figure  9.  Contrast  enhancement  may  be  a  primary  task  of  generalization  neurons.  Here,  several 
neurons  share  a  set  of  afferents  and  can  evolve  such  that  the  -cells  prefer  various  patterns  while  the 


G<clls  provide  inhibition  uniformly  across  the  environment,  thus  suppressing  partially  excited  S-cclls. 


This  circuit  permits  S-cells  to  attain  much  higher  selectivities  even  with  positive  semidefinite  input 
weights.  •  :  excitatory  synapses  ;  H  "inhibitory  synapses. 


( 


SCLCCTIVITY  MAXIMIZATION 


orientation 


FILMED 

3-85 


DTIC 


